Search Results: "jelmer"

1 June 2010

Steve Langasek: bzr-git case study

Robert Collins did a very nice writeup not so long ago of creating a Debian package with bzr-builddeb, but he starts from the assumption that upstream also uses bzr. As we all know, there are many projects that have their upstream sources in a VCS, but don't use bzr. Can we still get reasonable results from bzr-builddeb, including the ability to merge new upstream releases direct from the VCS, if, say, upstream is using git? Yes, we can! Of course, if you're happy with using git as the VCS for your Debian packaging already, there's probably no compelling reason for you to switch to bzr. But if you prefer to use bzr and have been frustrated at having to choose between being able to use the VCS client of your choice for your packages and being able to use your VCS client to access upstream revision history, read on. Thanks to the fine work of Jelmer Vernooij and friends, bzr-git has been usable for a while if you just want to track the default git branch. But what if you care about tracking multiple upstream branches? Enter bzr git-import, which we'll use to set up a bzr repo from scratch for our cifs-utils package, with a full import of all the branches of upstream's git tree. First we do some work to set up a shared bzr repository. As long as we're going to the effort of mirroring all the git branches, we probably want to publish them for other people to use, so we make sure our bzr repository is reasonably configured for sharing data between branches:
$ bzr init-repo --no-trees cifs-utils.server
Shared repository (format: 2a)
Location:
  shared repository: cifs-utils.server
$ cd cifs-utils.server
Then we use bzr git-import, provided by the bzr-git plugin, to do a full import of the upstream branches into a cleverly named subdirectory:
$ bzr git-import git://git.samba.org/cifs-utils.git upstream
[/                   ] Counting objects: 181, done.
Now we create our own local 'trunk' for the package, by branching from the upstream tag matching the release version we're going to package.
$ bzr branch -r tag:cifs-utils-4.0rc1 upstream/HEAD trunk
Branched 42 revision(s).
Do a little more prep work for future branches...
$ mkdir branches
And that's it. Now we have a bzr repo locally that we can push out to our hosting server of choice (N.B.: we could do this all directly on the server, but alioth doesn't currently have bzr-git installed):
$ rsync -az . alioth.debian.org:/bzr/pkg-samba/cifs-utils/
$
Now that we have a bzr repository, it's time to get ourselves a working directory and do some packaging. We're probably going to work with multiple branches locally as well, so we create another shared repository for local use, and get a copy of the trunk branch we created before.
$ bzr init-repo cifs-utils
Shared repository with trees (format: 2a)
Location:
  shared repository: cifs-utils
$ cd cifs-utils
$ bzr co bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/trunk
Now we grab the upstream tarball, and whip up some quick packaging with (what else?) debhelper 7:
$ wget ftp://ftp.samba.org/pub/samba/cifs-utils/cifs-utils-4.0rc1.tar.bz2
$ ln -s cifs-utils-4.0rc1.tar.bz2 cifs-utils_4.0~rc1.orig.tar.bz2
$ cd trunk
$ mkdir -p debian/source
$ echo '3.0 (quilt)' > debian/source/format
$ echo 7 > debian/compat
$ cp /usr/share/doc/debhelper/examples/rules.tiny debian/rules
$ dch --package cifs-utils --versio 4.0~rc1-1 --create 'Initial package'
Create debian/control by hand, generate an initial source package, and import it into bzr with bzr-builddeb.
$ debuild -uc -us -S -i
$ rm -r debian
$ bzr import-dsc ../*.dsc
$
Since we want to continue tracking upstream development, we need to periodically sync our bzr import of the git tree.
$ bzr git-import git://git.samba.org/cifs-utils.git bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/upstream
$
And when we find a new upstream version in that import that we want to package, bzr-builddeb makes this a snap, too.
$ cd cifs-utils
$ wget ftp://ftp.samba.org/pub/samba/cifs-utils/cifs-utils-$version.tar.bz2
$ cd trunk
$ bzr merge-upstream --v3 --version $version_with_epoch \
    ../cifs-utils-$version.tar.bz2 -r tag:cifs-utils-$version \
    bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/upstream/HEAD
No distribution specified, and no changelog, assuming 'debian'
Committing to: /tmp/tmpbzOVMs/upstream/
Committed revision 2.
All changes applied successfully.
The new upstream version has been imported.
You should now review the changes and then commit.
$ bzr diff
<review changes>
$ bzr commit -m "merge upstream $version"
Committing to: bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/trunk
modified debian/changelog
Committed revision 50.
$
And that's it! It's exciting to see signs of real interoperability between DVCSes at last. Thanks to everyone who's helped make it possible!

1 November 2009

Jeff Bailey: Crap error messages

updated

When writing software, *please* don't give error messages that are only meaningful to developers of the software. Microsoft used to be awful for this: "System fault at DEAD:BEEF, please contact your system administrator". Which would've been cool, except that I *was* the system administrator.

My most recent encounter with this was this morning with bzr (On my work laptop, so it's Windows):

C:\Users\jeffbailey\workspaces\webhack\Webhack>bzr push lp:~jbailey/webhack/trunk
Connected (version 2.0, client Twisted)
SSH C:\Users\jeffbailey/.ssh/id_rsa password:
Authentication (publickey) successful!
Secsh channel 1 opened.
bzr: ERROR: RemoteRepository(bzr+ssh://bazaar.launchpad.net/~jbailey/webhack/trunk/.bzr/)
is not compatible with
CHKInventoryRepository('file:///C:/Users/jeffbailey/workspaces/webhack/Webhack/.bzr/repository/')
different rich-root support


Umm. What? Okay. I work for a search engine. So I fired up Yahoo!^WGoogle to figure out what the message could possibly mean. The top result is Bug 172360 in Launchpad:
"branching from rich-root to non-rich-root gives confusing errors"

Sounds perfect, until you see that the bug is marked as invalid.

Something got missed here. If your user says "I don't understand" - they are NEVER wrong. It doesn't matter if you think they should've understood. It doesn't matter that you think you explained it well enough. The user doesn't understand, so you lose. Actually, it's worse. *they* lose.

There are lots of other bugs leading up to how I got to where I was seeing this in the first place. But none of those are as important as the fact that the developers thought a user saying "I don't get it" wasn't worth the 30 seconds of adding the text "The best way to fix this is to run: bzr upgrade --default " + branch.url;

Update: Jelmer pointed out to me in a comment that it wasn't the devs who marked it as invalid, it was the submitter. He's reopened it. This just further supports the notion that users are insane and need hand-holding. =)

24 October 2009

Obey Arthur Liu: Debian at Google Summer of Code Mentor Summit

Debian at Mentor Summit

From left to right: Obey Arthur Liu, Olly Betts, Stefano Zacchiroli, Dirk Eddelbuettel, Sylvestre Ledru, Jelmer Vernooij.

Dear Planet,

We arrived at the Google Summer of Code 2009 Mentor Summit and are having a blast here. The weather is awesome, the candies are plenty and the conference rooms are comfy at the Googleplex. We will write to you again soon.

Cheers

The Debian people Arthur, Olly, Zack, Dirk, Sylvestre, Jelmer

13 September 2009

Jelmer Vernooij: CtrlProxy: Looking for a new maintainer

After over 7 years of working on it off and on, I'm looking for somebody to help maintain (and eventually take over) CtrlProxy. I started working on CtrlProxy somewhere in 2002, only a short while after Wilmer started hacking on BitlBee. If I remember correctly I started working on it because I didn't want to run a separate dircproxy (the only real competitor at the time) instance (with configuration) for each IRC network that I connected to. It was also just a good excuse to play with the IRC protocol a bit. Over the years, CtrlProxy has served as a playground for me to try out new and interesting things. It's been rewritten or severely refactored several times in its early history, the latest time being the 3.0 release (from 2005). I've tried different build systems, I've tried different implementation languages, I've tried different configuration file formats, I've tried different support libraries, I've tried different version control systems, I've tried different documentation formats. So while it's definitely been a very educational project for me personally, I haven't really had the time or the interest to dedicate to the project that it deserved during the last few years. This was mostly because there were other more interesting FOSS projects I spent my spare cycles on. These days there are plenty of other good IRC proxies out there, including BIP, ZNC and ShroudBNC so I doubt CtrlProxy will be missed if it were to disappear. Despite that, if anybody is interested in taking over, please send me an email (jelmer@samba.org) or contact me on IRC (jelmer on the OFTC and Freenode networks). cp: Anathema - Shroud of False1

7 June 2009

Sam Hartman: Debconf and Debcamp

I will be attending Debconf 9 in Spain from July 23-30. I will also be attending debcamp the previous week. I m hoping to build contacts and increase my involvement in the Debian community, and the previous debconf I attended was an interesting window into what was going on in Debian and Linux. I m still lining up things to do at Debcamp. Jelmer Vernooij will be there; he s interested in working with me on Samba 4 support for MIT Kerberos in Debian. I m interested in working with him on making the user experience good for people who use both Samba 4 and other Kerberos applications. As I wrote at the bottom of this post, I believe it is critical that the open source community not just follow what Microsoft is doing in the Enterprise space. I also think it is important that we maintain avenues for our own innovation. To that end, I want to look at what we can do to use enterprise infrastructure independent of AD-look-alike projects like Samba as well. So, I ll be looking at making what I can do to help this in Debian. Areas of interest include:
  1. Easy set up of Kerberos to use an LDAP database
  2. Easy configuration of libpam-krb5 and libpam-ldap together using Kerberos for authentication and LDAP for authorization but not authentication.
  3. Support for FAST integrated into Debian systems so we can gain better protection against weak passwords. As I promised, more about this in its own post.
  4. Better support for PKI/smart cards for network authentication.
These are all projects I think I could make headway on myself. However the value of debcamp is the other people there. I ve never been to a debcamp before and so I don t know what it will be like. I do know that I will give higher priority to projects that will benefit from close cooperation over a week. So, if you re there and want to try to recruit me to your project, feel free. I m interested in enterprise infrastructure, VOIP, IPv6, network security and making complex infrastructure easy to use.

24 May 2009

Christian Perrier: News from the samba packaging team

Some (not so) short news from the samba packaging team... Recent package uploads: Other things are coming, such as the work on having samba3 and samba4 packages coexist. Jelmer Vernooij prepared everything after our discussions at the SambaXP conference and it should be uploaded "soon". A few patches we have should be integrated upstream as well, therefore making the Debian diff smaller and smaller. Also, Luk Claes joined the maintenance team and helped setting up the unofficial repository for backports. The bug count is going up again these days as I don't spend much time triaging the bugs and some are very tricky to reproduce. As usual, help would be welcomed. I'm fairly sure that several of these are user errors but I often lack time to prove this enough for closing the bug. We should go to to 50-60 bugs or so.

29 September 2008

Christian Perrier: The best way to get bug triage...

... is to have upstream do it..:-) Apparently, yesterday, Jelmer Vernooij, member of the Samba Team and maintainer of samba4 and related packages (openchange, etc.) started some triaging of bugs of the samba package. After several years of efforts by Steve Langasek, No l K the and /me, the bug log of samba is not in a so bad shape, but, still, tseeing one of our upstream digging in our dirt is very good news... Thanks again, Jelmer. You really deserve to reach the end of your NM process, now.

9 June 2008

Christian Perrier: Samba 4 in experimental

The (still alpha) 4.0.0 release of Samba is now in experimental. Both Samba 3 and Samba 4 packages are likely to coexist in the archive as Samba 3 is still for a while the production code for File and Print services while Samba 4 is the development branch of the Samba team to implement Active Directory domain control (among other things). Samba4 packages also bring a big bunch of libraries, most of which being used to related development such as Openchange. Expect some neat new packages in experimental, then unstable, pretty soon now. Please do NOT replace production servers running samba 3.0.* or samba 3.2.* by samba4....not yet..:-) The samba4 packages are maintained by Jelmer Vernooij, one of the two main developers for Samba 4 in the Samba Team (the other being Andrew Bartlett). Jelmer is someone we should stuck out from the NM queue as soon as possible..:-) Anyway, this is again another success of that small team we built around Samba and samba-related packaging. Thanks to all involved people (Steve Langasek, No l K the, Jelmer Vernooij, Peter Eisentraut, etc.).

31 May 2008

Christian Perrier: News from samba packages development

There has been some hype around samba this week. On May 21st, the Samba Team released Samba 3.0.29, an update from their stable branch. The Debian package maintainers quickly built and uploaded a package for unstable/lenny the very same day. Then, on 27th, we got notified of CVE 2008-1005, which affects both etch, lenny and unstable (not the 3.2 versions we have in experimental). I contacted the security team and, after some brief discussions, I uploaded with their blessing, a fixed package to stable-security (3.0.24-6etch10). Then, I immediately uploaded samba 3.0.30 to unstable (1:3.0.30-1), that version being the Samba Team response to the security issue. All this lead to DSA 1590. Up to now, everything perfect. In the meantime, Karolin, the Samba Team's release manager announced the Releace Candidate 1 of Samba 3.2.0. So, yesterday (Saturday), I started building it....which succeeded. Then I started to screw up..:) Instead of uploading that version to experimental, as planned, I uploaded 1:3.2.0~rc1-1 to unstable. Yes, AGAIN (I already did that in April). There's no excuse for that, except distraction and failure to use our tools properly ("dch -r"). Thankfully, I noticed that nearly immediately, but not immediately enough. So, I had to re-upload 3.0.30 to unstable (which is urgent: that's a security fix) and increase the epoch (we already had one as this is the second identical screwage): 2:3.0.30-2. Sorry, autobuilders... Yesterday evening, I uploaded 3.2.0-rc1 again to experimental (2:3.2.0~rc1-2). Then, finally, I went on the last bit of remaining work: spend some time on Jelmer Vernooij's samba4 packages. Jelmer is one of the two lead developers of Samba 4, a huge masterpiece still under heavy delveopment. To give it more exposure, it is planned to upload it a few times in experimental, then move it to unstable, block it so that it does not enter testing, and make some noise about this. I finally uploaded samba4 this morning (triple checking that I was uploading to experimental!) and samba4 4.0.0~alpha4~20080522-1 is now waiting for NEW processing (which might take time: this is a big piece of new stuff). I'm sad for the screwage because, apart from it, that was nearly a perfect process. But, anyway, I'm still proud of the result: 4 successful uploads for samba in a little more than a week (which included 4 day away). It could even have been 5 as the samba version in sarge is affected by the security issue, but we don't support sarge anymore.

19 April 2008

Christian Perrier: Samba week

I spent the entire week at the SambaXP conference in G ttingen, Germany. The conference is the annual conference of the community of developers, contributors and users of Samba. I general call it "my annual german pilgrimage" as I attended all seven editions of the conference, the only FLOSS conference I attend on my work time, as my daily work involves quite a lot of uses of samba. This year featured a workshop or training session held by John Terpstra, longstanding FLOSS evangelist and member of the Samba Team for over 10 years, and Karolin Seeger, the brand new release manager of Samba. This has indeed been an incredible opportunity to have discussions with them about the packaging work for Samba. Actually, the work we did in the last 3 years to bring Samba Debian packages in a very good shape (when I don't screw up) is much appreciated. I think these days have been another opportunity to keep that link very closed. I have been impressed by the promising work of Karolin with respect to the preparation of the release and the very serious way she has to do that work....as well as her very friendly, while still discreet approach to technical discussion. Some work was done on Samba package bugs, though less than usually (the remaining ones are harder and harder to tackle!). I mostly work on Debian packages as well as .deb packages prepared by Sernet (the services company that organises the conference and employer of some Samba Team members), as these could some day become the packages provided by the Samba Team. Talks at the conference were pretty interesting, to keep connected with Samba 3.2 and 4.0 development. News from Andrew Tridgell about exchanges with Microsoft (yes, The Evil) and access to MS documentation, are very promising. The collaboration between Samba developers and Microsoft enginneers is now working well again, at the engineers level (as Tridge says: "lawyers are away, now, we can talk at the engineers level and restore the link that existed in the early 90's"). I also could measure the progress of the Openchange project whose ultimate aim is to provide a complete Microsoft Exchange replacement solution. They currently have working MAPI libraries and an Evolution plugin for Exchange is under development, while the bricks to build a server are patiently being put together. Good discussions with Julien Kerihuel, the lead (and French) developer and manager of the project. Jelmer Vernooij and I also settled the final plans to get Samba4 packages in Debian. I proposed to make a first upload to experimental, but quite soon to upload to unstable, the point being a largest as possible exposure of that code, so that upstream developers (Jelmer himself, Andrew Bartlett and a few others...) get as much feedback as possible. That upload will not be targeted for lenny (we'll block it from entering testing). For that reason and also because Samba 3 and 4 will certainly coexist for years, the source package will be named "samba4". Expect more news quite soon. And final conclusion of that week: I've also been delighted to be able to visit Alex and Meike in Hildesheim, as well as Andreas in Wernigerode. It's always a great pleasure to see longstanding Debian friends. At the beginning of the week, I planned to see *two DD and finally, at the end of the week, I can tell I've seen *three* Debian developers, finally!

14 July 2007

Robert Collins: 14 Jul 2007

So we're here in sunny Vilnius sprinting on bzr. I thought I'd write up some of what we've achieved. On Thursday Wouter got most of the nested-trees branch merged up with bzr.dev, but about 500 tests failing ;). Jelmer has introduced a new parameter to the sprout api call on BzrDir called 'limit_to_revisions' which if supplied is a set of revisions which forms a strict limit on the amount of data to be fetched. This is intended to be the API by which the desire for limiting history - to achieve shallow branches/history horizons - is communicated at the 'bzr branch' or 'bzr checkout' stage. This API is tested so everything passes, but nothing chooses to implement it yet. I was hacking on commit (somewhat indirectly) by working on the repository-wide indexing logic we will require if we wish to move to a more free-form database with semi-arbitrary keys mapping to opaque data (at the lowest level). I got a core GraphIndex up at this point. Once complete this should drop the amount of I/O we perform during commit by half. There was some other hacking going on too - Tim Hatch did some tweaks to bzr-hg, Jelmer some bzr-svn, and so on. On Friday we carried on on the same basic targets: Wouter fixed the 500+ failing tests so now its only in the 5-10 range, and has been debugging them one by one since. Jelmer implemented the limit_to_revisions parameter all the way down to the knit level, for non-delta compressed knits (e.g. revisions.knit), which made knit1 repositories support the parameter and got the 'supports' branches of the interface tests being executed. I developed CombinedGraphIndex, and have sent the low level Index stuff up for review, and followed that up by implementing a KnitIndex implementation that layers on the GraphIndex API to allow the use of .knit files without .kndx. This new index API allows us to switch out the indexing code and format much more easily than previously - the knit index logic is decoupled from the generic indexing facilities. It also allows us to record arbitrary-parent compression chains. Finally Jelmer implemented the wishlist item for bzr-gtk of a notification area widget that can be used to enable bzr-avahi watching, and share commits with the local LAN/get notified of commits on the local LAN. Today is the last day of the sprint, and I hope that we'll get the nested tree branch passing all tests, the limit_to_revisions parameter causing delta-compressed knits to only copy data outside the listed parameters to the first full text, and have an experimental repository format that uses the new index layer for at least a couple of its knits (e.g. revisions and inventory, or revisions and signatures).

25 April 2007

Christian Perrier: SambaXP

This is now established as my german annual pilgrimage. Every year since 2002, I migrate to G ttingen for 3 days to attend the SambaXP conference, the world conference of the users and developers of Samba. Halfway between a very classical conference (high-class hotel, nice conference rooms, drinks and food at will...and expensive fee) and a "geeky-style" conference (the whole Samba Team usually attends it as well as many users and developers), it is now established as the annual reference for people involved in Samba use and development. After an always interesting tutorial (this year, as often by Jerry Carter, Samba3 release manager) about making samba friendly to administrate to....Windows admins, the track continued with talks drawing the future of samba development. This year, I was however missing No l K the who could not attend, so the Debian crowd here is pretty small (but many Debian users and admins are around). I however could have interesting talks with Jelmer Vernooij (mostly involved in Samba4 development...and also in our NM queue). I also used some time to try cleaning out the Debian BTS for samba bugs, using the opportunity to show old bugs to upstream developers. I also could visit Alex and Meike who live not that far, which was a real pleasure, including the Real Swabian Sp tzle that make swabian women so strong and powerful. The conference will end up today. It will have been too short, as usual and I will feel like I'm running away on the A7 up to Hannover to catch the plane and come back home....and, as usual, I'll promise myself to come back next year and practice my German, which sucks. Tomorrow, leaving to Rome for a long week-end with Elizabeth. No laptop, no mail. Just enjoy the city. Gut!

3 February 2007

Robert Collins: 3 Feb 2007

Speeding up bzr info on the hpss with a good sized branch - samba 4.0 (which is > 200MB of bzr repository).
bzr info bzr+ssh://host/home/robertc/SAMBA_4_0
Location:
  branch root: bzr+ssh://host/home/robertc/SAMBA_4_0/
 Related branches:
  parent branch:
http://people.samba.org/bzr/jelmer/mirror/samba/branches/SAMBA_4_0/
 Format:
       control: bzr remote bzrdir
        branch: Remote BZR Branch
    repository: bzr remote repository
 Branch history:
     11316 revisions
      1035 days old
   first revision: Sun 2004-04-04 08:56:30 +0000
  latest revision: Wed 2007-01-24 12:23:42 +0000
 Revision store:
     11316 revisions
    111019 KiB
Before our latest optimisations:
real    14m21.769s
user    1m23.377s
sys     0m11.017s
After:
real    0m11.203s
user    0m0.632s
sys     0m0.104s
I say 'whoot' to the sprinters!

5 October 2006

Thijs Kinkhorst: BSP a success

The recent Debian Bug Squashing Party was quite a success if you ask me. The most problematic was that it coincided with the Utrecht Programming Championship (a qualifier for the ICPC NWERC), which took much of my time as an organiser and of Jeroen and Jelmer as contestants. It was worthwhile though, because their Team J won the contest. On the BSP a significant number of bugs have been squashed, but also a lot of work has gone into debian-cd, the release notes, security updates and removing packages. Steve has very few photo's available. Let's repeat it for etch+1, especially when it doesn't share a weekend with other local events.

27 April 2006

Christian Perrier: Back from my annual German samba break

As usually every year, I went this year to the Samba eXPerience conference in G ttingen, Germany. That conference is the only existing conference entirely devoted to the Samba project. It is a unique opportunity to meet with other users of samba, as well as nearly all member of the Samba development team (which is "only" made of 10-15 people, which always seem to surprise attendants when one knows about the quality of Samba over the years). This year, we were missing Tridge who was busy in European Commission hearings about the Microsoft case, if I got the explanations correctly. But most of others were around and I'm glad I had a few really interesting talks with several key people in the Samba project. The small samba maintenance team in Debian will probably take advantage of this as well. There were two of us over there, No l K the and myself. It's always a pleasure to spend some time with No l. We've been able to lower the number of bugs for samba in Debian a little bit as well as prepare the next release (3.0.23) packages...finding a few bugs in the upstream builds..:-) We also initiated some discussions with upstream about possible backports of the current samba code to sarge. Upstream policy is to encourage users to upgrade to the last version in the samba3 releases, so our policy in Debian does not fit well with these goals. We'll try to work closely with people who maintain the Debian packaging in the Samba Team... Of course, we also talked and listened to numerous talks about Samba 4, the Active Directory killer. In short they can be summarized to: nothing will be released "production ready" in 2006. So, we won't have samba4 in Debian etch. Steinar Gundersson and Jelmer Vernooj (member of the Samba Team AND in the Debian NM queue) will continue to maintain samba4 package in experimental (to test them, just install samba from experimental...beware, this *will* break File and Print services on your host). Also important were talk about evolutions in Samba3, the current production code. My vote goes to Jeremy Allison's talk about "Delegating administration of samba servers", introducing what I consider a killer features: user shares, ie the possibility to allow some users to create their own share on Samba server. Believe me, that feature will rock and it *will* be in 3.0.23 due out in a few weeks. I already have tons of plans for my production server at work in ONERA where the days of my NT4 domains are counted. For this, Jerry Carter tutorial about Samba 3 and Active Directory will be incredibly useful, even if I don't have any AD servers (and don't plan to). In short, great time as usual in G ttingen. I'm also very happy that I had the opportunity to visit Meike and have such long and interesting hours talking together while eating noodles. Tsch ss!

Next.

Previous.